Watermarking Encoded Content

ABSTRACT

A system and method are described for inserting a watermark in encoded content, including receiving encoded content, receiving at least one pre-processed watermark unit, and replacing directly a number of bits starting at a position in the encoded content with an alternative value, wherein the alternative value has embedded therein watermark signals, wherein the alternative value is selected from among a plurality of alternative values specified in the at least one said watermark unit. An apparatus and method for generating watermark units are also described including selecting a position where a value in encoded content is to be replaced by an alternative value and calculating the alternative value a plurality of alternative values for each selected position. Further a system and method for replacing a watermark in encoded content are also described including receiving encoded content, receiving at least one pre-processed watermark unit and replacing a number of bits starting at a position in the encoded content with a value, wherein the value is specified in the at least one watermark unit.

FIELD OF THE INVENTION

The present invention relates to digital watermarking and specifically,to embedding, removing/replacing and detecting digital watermarks inencoded content.

BACKGROUND OF THE INVENTION

Encoded content requires a “decoding” process in order to use thecontent. Some examples of encoded content include MPEG-1 MPEG-2H264/AVC, WMA, MPEG4, JPEG2000, MP3, PDF, Windows Word, Postscript, etc,and their encrypted versions.

One prior art method for watermarking encoded content is to insertwatermark signals in the structure and syntax elements. U.S. Pat. No.6,687,384 is an example for embedding data in syntax elements in a codedbit stream such as MPEG-1 and MPEG-2. Such watermarks do not, however,survive format change or digital-analog conversion.

Another prior art method for watermarking encoded content is to embed awatermark by adding noise to the DCT coefficients for MPEG encodedcontent. An example of this approach is described by F. Hartung and B.Girod in “Digital Watermarking of MPEG-2 Coded Video in the Bit StreamDomain”, Proc. IEEE ICASSP, pp. 2621-4, April 1997. The method ofHartung and Girod does not use perceptual techniques.

Yet another prior art method for watermarking encoded content is todirectly modify the encoded content. U.S. Pat. No. 6,373,960,hereinafter “Conover”, describes a watermarking method within an MPEGcompressed video bit stream by modifying some DCT coefficient in such away that the entropy encoded length of these coefficients remainunchanged after modification. No specific watermark embedding methodsare specified in Conover. The methods for selecting watermark sites inConover are limited to the coefficients with 0 run-zero. Thecoefficients of Conover are in the high frequency domain. The method ofConover includes no preprocessing phase nor is there any “alternativevalue” for actual watermark insertion in the later phase described. Theconcept of generating and using “watermark units” is nowhere disclosedor taught in Conover.

The prior art methods for watermarking encoded content does not have apre-processor for producing “alternative values” for some parts of theencoded content. The prior art methods for watermarking encoded contentexclude encoded content that is further masked, obfuscated, scrambled orencrypted (collectively “encrypted”).

SUMMARY OF THE INVENTION

The present invention watermarks encoded content by embedding watermarksin selected locations in the encoded content. This is accomplished byreplacing original values in the encoded content by their alternativevalues. Each original value may have one or more alternative values andeach alternative value contains a watermark signal. Moreover, watermarkscan be removed by replacing the alternative values with the originalvalue, or by substituting one alternative value with another alternativevalue.

By selecting one of the alternative values to substitute for theoriginal value in the corresponding locations in the encoded content,the following goals are fulfilled:

-   -   The alternative values are identical in size to the original        value, and replacement by any one of the alternative values        creates an encoded content compliant with a defined format and        does not introduce perceptual artifacts in the encoded content.    -   The alternative values may improve the quality of the encoded        content. For example, the original value may or may not be a        valid value. In other words, without the replacement by one of        its alternative values, the encoded content may not be a        legitimate format. In another case where the original value is a        valid value, without the replacement by the alternative value,        the original value may introduce the degradation of the encoded        content.    -   Each alternative value contains watermark signals. These        watermark signals, either together with other watermark signals        at other positions within the encoded content or by themselves,        can have one or more units of information embedded. A unit of        information consists of one or more bits.    -   The data volume of the alternative values (defined by the data        size per alternative value and the number of alternative values)        must be small, compared to the data volume of encoded content.

One important goal of directly watermarking encrypted content is“localized encryption”. A localized encryption allows for acorrespondence between the clear text and cipher text. For example, aunit of clear text includes components c1,c2, . . . cn. After localizedencryption, the cipher text consists of c1′, c2′, . . . , cn′ where ci′is the encrypted version of ci (1≦i≦n). A simple example of localizedencryption is to split the content into parts and encrypt each partseparately. For example, in an extreme case, each coefficient or groupof coefficients is encrypted separately. One category of encryption,which makes localized encryption possible, is selective encryption orpartial encryption. Instead of treating content (audio or video) asbinary data streams (also called “native encryption”), selectiveencryption methods “understand” the content syntax structure (e.g.MPEG-2 structure) and only selectively encrypt some parts of thecontent. Selective encryption occurs after compression. Encryptedcontent must not have commercial value, although part of content may bevisible. Some selective encryption schemes may preserve the format, bitrate, and size of the encoded content that is not selectively encrypted.Other selective encryption schemes may increase bit rate or require aspecial decoder.

Taking MPEG-2 content as an example, a simple selective algorithmencrypts only the I-frames. The selectively encrypted MPEG may be avalid MPEG stream. Although P and B frames in MPEG-2 are not valuablewithout knowledge of the corresponding I frames, large portions of MPEGvideo are still visible because of inter-frame correlation and mainlyfrom unencrypted I-blocks in the P and B frames. Other selectiveencryption schemes include encryption of MPEG-2 headers and/orencryption of DCT coefficients. The DCT coefficients are divided into“DC coefficient” and “AC coefficients”. DC coefficients are thecoefficients with zero frequency in both dimensions, and AC coefficientsare the remaining coefficients with non-zero frequencies. All DCcoefficients or partial values of AC coefficients of all I-blocks can beencrypted.

A system and method for inserting a watermark in encoded content aredescribed including receiving encoded content, receiving at least onewatermark unit and replacing directly a number of bits starting at aposition with an alternative value, wherein the alternative value hasembedded therein watermark signals. A apparatus and method forgenerating watermark units are also described including selecting aposition where a value in encoded content is to be replaced by analternative value and calculating the alternative value. Further asystem and method for replacing a watermark in encoded content are alsodescribed including receiving encoded content, receiving at least onewatermark unit and replacing a number of bits starting at a position inthe encoded content with a value, wherein the value is specified in theat least one watermark unit and further wherein the position isspecified in the at least one watermark unit.

Further a system and method for inserting a watermark in encoded contentare described including receiving encoded content, receiving at leastone watermark unit, receiving a sequence of bits, wherein the sequenceof bits comprises watermark payload information and performing one ofreplacing directly a number of bits starting at a position with analternative value and leaving unchanged the number of bits starting atthe position based on bit values of the sequence of bits of thewatermark payload information and wherein the alternative value hasembedded therein watermark signals. A system and method for watermarkdetection are also described including receiving at least one watermarkunit, receiving watermarked content, retrieving a plurality ofcoefficient values from the watermarked content and retrieving bitvalues of the watermark from the coefficient values.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is best understood from the following detaileddescription when read in conjunction with the accompanying drawings. Thedrawings include the following figures briefly described below:

FIG. 1 is a block diagram showing the workflow for the pre-processingfor watermark embedding in accordance with the principles of the presentinvention.

FIG. 2 depicts a watermark unit (U) of the present invention.

FIG. 3 is flowchart of the method of watermark detection in accordancewith the principles of the present invention.

FIG. 4 is a block diagram of a playback device, which receives encodedcontent having watermark units embedded therein.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Watermarking encoded content can be divided into three separate steps:

-   Location selection, which is the selection of the position where the    value in the encoded content can be replaced by alternative values    containing watermark signals.-   Calculation of alternative values, which is the determination of the    alternative values so that the alternative values have the same    number of bits as the value the alternative values are to replace in    the encoded content and such substitution will not cause perceptual    changes to the content. Moreover, these alternative values contain    watermark signals.-   Watermark embedding/insertion, which is the actual replacement of    the value in the encoded content by one of the alternative values.

The first two steps can be pre-processed. As a result of thepre-processing, a set of watermark units (WUs) are produced. WUs containall information for actually embedding/inserting a watermark. Thewatermark pre-processor takes encoded content as the input and awatermark key, and outputs a sequence of watermark units. In the casewhere only one alternative value is generated for each WU, a watermarkpayload may become an additional input to the pre-processor. Thesewatermark units are integrated into the final content as meta data ofthe encoded content, as a separate channel multiplexed with the content,as steganographic data hidden in the syntax elements or in the content,or as a separate file stored in the physical media (optical disc, tape,hard drive, etc.) or transmitted over a network (TCP/IP, satellite,etc.).

The watermark payload information is embedded by selecting from amongthe alternative values in WUs. A WU has at least one alternative value.When each WU has only one alternative value, there are two methods toembed/insert the watermark information (payload). The first method is toembed single fixed watermark payload by replacing all original values inWUs by their alternative values. The second method allows for embeddingvarious watermark payloads by switching between replacement ornon-replacement for WUs. For example, a replacement indicates a positivebit value and non-replacement for the negative bit value. To embed awatermark payload ‘00101001”, the original values in the first andsecond WU are not replaced, the original value in the third WU isreplaced by its alternative value. For a WU with more than onealternative value, each alternative value, which contains a watermarksignal that may represent a different information unit of watermarkpayload. The alternative value is selected for substitution based on thewatermark payload. For example, to embed one bit in a WU, only twoalternative values V1 and V2 are required. To embed a bit value ‘0’, V1is selected to replace the original value in the encoded content, and toembed a bit value ‘1’, V2 is selected. With two values V1 and V2 it ispossible to express a ‘0’- and a ‘1’-bit. If there are four alternativevalues (V1, V2, V3 and V4) two bits information (i.e. ‘00’, ‘01’, ‘10’and ‘11’) can be embedded. With more alternative values at one position,more bits can be embedded, which allows for a very efficient embeddingof information.

The watermark payload is typically received in the third step “WatermarkInsertion”. It can be stored in or calculated by a component external tothe watermarking system. The watermark payload information is typicallyan identifier uniquely identifying the recipient, playback device(manufacturer, model and/or serial number), or date and time of theplayback of the content. The difference between the original value, V,and its alternative values may be stored in WUs to potentially reducethe size of WUs. WUs can be further compressed.

It is important that the WUs are protected against unauthorized accessor modification, because with this information not only the watermarkingsystem is vulnerable to various attacks but also it is easy for a hackerto insert fake watermarks, or alter or remove existing watermarks. Ifthe WUs are stored and transmitted as steganographic (watermarking)data, the access can be controlled by a watermarking key. If the WUs arestored and transmitted over any another channel, it requires encryptionof the WUs.

As indicated earlier, the first and second steps can be performed as apre-processing. Thus, prior to the third step the encoded content is notwatermarked. The third step performs the watermark embedding byreplacing some values in the encoded content with the alternative valuesspecified in WUs. These values can be encoded syntax elements such aspackage identification, headers, quantization table, Huffman table forentropy coding, encoded coefficients or encoded motion vectors. The WUsspecify where the watermark signals (in alternative values) are embeddedand what the possible watermark signals can be in these positions byselecting the appropriate alternative values.

FIG. 1 is a block diagram showing the workflow for preprocessing ofencoded content to generate WUs. FIG. 2 depicts a watermark unit inaccordance with the principles of the present invention. Each WU isdescribed by a vector (P, L, C, {V}, V1, V2, . . . Vn) where P is theposition of the original value V that may be replaced in the future byan alternative value in the encoded content, L is the number of bitsoccupied by the original value V in the encoded content starting at P, Cis a set of global positions of the coefficients (for example, the DCTor wavelet quantized coefficients) that are encoded in the L bits (inencoded content) with an entropy coding method. Entropy coding such as aHuffman coding is typically applied to the last stage of encoding toproduce the encoded content. V1, V2, . . . Vn are valid alternativevalues of V and each of such values contain a watermark signal. Eachcoefficient position is represented by c(ch,f,b,co) where ch is thechannel index, f is the frame index, b is the block index, and co is thecoefficient index within the block for a unit of video content. Analternative value is valid, if, when this value replaces the currentvalue it maintains format compliance and there are no perceptual effectson the content. Moreover, V1, V2, . . . Vn occupy L bits as V does inthe encoded content. {V} indicates that the original value is optionalin the embedding/insertion process. The original value may be requiredin the watermark removal processes.

An original value V may contain one or more coefficients in encodedform. When the WUs are generated, if the input is encoded content suchas MPEG-2 or MPEG-4, the entropy coding, i.e. Variable Length Coding(VLC), first needs to be “undone” to access to coefficients, and thenfind the appropriate position and alternative values and store theoriginal coefficient and these alternative values. An alternative valueVi may correspond to the same coefficients as V does, but in some casesVi may correspond to more or less coefficients than V does.

A typical 8×8 block of quantized DCT coefficients in MPEG-2 is shownbelow. Most of the higher order coefficients have been quantized to 0.

12 34 0 2 0 0 0 0 87 0 0 1 0 0 0 0 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

After zig-zag scanning the sequence of DCT coefficients to betransmitted looks like:

-   -   12 34 87 16 0 0 2 0 0 0 0 0 0 1 0 0 0 0 0 0 . . .

The first DCT coefficient (12) is sent via a separate Huffman table.After Run-Level parsing, the remaining coefficients and associated runsof zeros are:

-   -   34| 87 | 16 | 0 0 2 | 0 0 0 0 0 0 1 | 0 0 0 0 0 0 . . .

Using the DCT Coefficients Table Zero listed below and specified inMPEG-2 standard, these coefficients are encoded into 6 Variable LengthCodes (VLC) as follows (in binary bit representation):

-   -   0000 0100 0010 0010 | 0000 0100 0000 0000 0101 0111 | 0000 0000        0111 110 | 0000 1000 | 0001 010 110 | 10

MPEG-2 DCT Coefficients Table Zero (Part)

Variable length code (VLC) Run Level 10 End of Block . . . . . . . . .0001 10 s 1 2 0001 11 s 5 1 0001 01 s 6 1 0001 00 s 7 1 . . . . . . . .. 0000 100 s 2 2 . . . . . . . . . 0000 01 Escape . . . . . . . . . 00000000 0111 11 s 0 16  . . . . . . . . . (note: the last bit ‘s’ in eachVLC above indicate the sign of the level: ‘0’ for positive and ‘1’ fornegative value)

-   -   MPEG2 Encoding of Run and Level Following an ESCAPE Code

Fixed length code Run Fixed length code signed_level 0000 00 0 1000 00000000 reserved 0000 01 1 1000 0000 0001 −2047 0000 10 2 1000 0000 0010−2046 . . . . . . . . . . . . 1111 1111 1111   −1 . . . . . . 0000 00000000 Forbidden . . . . . . 0000 0000 0001   +1 . . . . . . . . . . . .1111 11 63  0111 1111 1111 +2047

For ease of reading, the above VLC (also called entropy codes) areseparated by a “|”. The first and second DCT coefficients use a fixedlength code to encode Run and Level according to the table “MPEG2Encoding of run and level following an ESCAPE code”, and the remainingDCT coefficients are coded according to “MPEG-2 DCT coefficients Tablezero”. The last VLC is a special “End of Block” code indicating theremaining coefficients in the block are 0.

An alternative value can be any part of the above encoded bits. Forexample, one alternative value may consist of part of VLC, one or moreconsecutive VLCs, or one VLC plus part of next VLC.

In the following example of WU, an alternative value consists of oneVLC, which is the 5^(th) VLC:

-   -   (PO+63, 7, C, 0001010, 0001110, 0001100, 0001000)        Each element in this WU is explained as follows:    -   P0 is the starting position of this block in the encoded        content. “63” the relative position (in bits) to the beginning        of this block, 7 indicates a length L, C consists of 7        coefficients as follows:    -   c(f,ch,b,7), c(f,ch,b,8), . . . c(f,ch,b,13) where f is the        current frame index, ch is the current channel index and b is        the current block index of this block in the example.    -   The size of the alternative value is 7 bits, i.e. L=7.    -   0001010 is the original value (V), there are three alternative        values: 0001110 (V1), 0001100 (V2), and 0001000 (V3). These        alternative values are coded as the following Run and Level        values (refer to MPEG2 DCT coefficients Table zero),        respectively:        -   0001110: Run=5, Level=1        -   0001100: Run=1, Level=2        -   0001000: Run=7, Level=1

A Run (R) and Level (L) pair represents a sequence of numbers beginningwith R zeros followed by L. For example, a Run-Level pair (5, 3), i.e.Run=5 and Level=3, represents 0 0 0 0 0 3. Thus, the alternative valuesabove represent the following coefficients before entropy coding:

-   -   0 0 0 0 0 1 0 (Run=5, level=1)    -   0 2 (Run=1, Level=2)    -   0 0 0 0 0 0 0 1 (Run=7, Level=1)        Because the 5^(th) VLC is the last VLC coding non-zero        coefficients in this block, the alternative values do not        necessarily correspond to the same coefficients coded by V. In        case of V2 and V3, instead of coding seven coefficients in V1,        V2 encodes only 2 coefficients and V3 encodes 7 coefficients.

If the first alternative value in the example WU is selected by thewatermark inserter in the Step 3, this block will be changed to thefollowing quantized DCT coefficients:

12 34 0 2 0 0 0 0 87 0 0 0 0 0 0 0 16 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Another example of encoded content is the content encoded with H.264(MPEG4 Profile 10). There are some major differences in entropy codingbetween MPEG-2 and H264 including:

-   -   H264 supports both CAVLC (Context-Adaptive Variable Length        Coding) and CABAC (Context-Adaptive Binary Arithmetic Coding).    -   Instead of 8×8 DCT block in MPEG2, H264 may also use 4×4 DCT        block    -   Unlike AVLC (Adaptive Variable Length Coding) in MPEG-2, CAVLC        uses smaller VLC tables by coding levels and runs separately.    -   Instead of the fixed VLC tables in MPEG-2, VLC tables can be        switched according to the context information.

Because of the context-based entropy coding in H264, it is harder thanMPEG-2 to predict its encoded value after a coefficient is modified.However, one may force a H264 encoder to reduce such context-basedcoding during encoding. For example, an encoder may use a fixed NumTrailtable (e.g. NumTrail Table 0), instead of a NumTrail table based oncertain properties of neighborhood blocks. Another example is to use afixed length code xxxxyyy to encode the number of non-zero (NumCoeff)and the number of trailing ones (T1) for a 4×4 quantized DCT block wherexxxx is for coding the number of non-zero (0-16) and yy is for codingthe number of trailing ones (0 . . . 3). One can also force an encoderto use 4×4 block only. In the following example, some of the above rulesare applied for encoding a 4×4 block.

1. A 4×4 qualitized DCT coefficients:

0 3 −1 0 0 −1 1 0 1 0 0 0 0 0 0 0After zigzag scanning, the coefficients in this block:0 3 0 1 −1 −1 0 10 . . . 02. CAVLC coding consists of the following five steps:

-   -   a. Encoding the number of non-zero (NumCoeff) and the number of        trailing ones (T1): (5,3). The maximum number of T1 is limited        to 3. These two numbers (NumCoef and T1) are coded into the bits        “0001011” by using a 17×4 Huffman table (see below) (NumCoef:        0-16, T1: 0-3).

NumTrail 17×4 Huffman Table

T1 NumCoef 0 1 2 3 0 — — — — 1 000011 01 — — . . . . . . . . . . . . . .. 5 0000000111 000001010 000000100 0001011 . . . . . . . . . . . . . . .16 00000000000000000 000000000000001001 00000000000000100010000000000000010000

-   -   b. Encoding the signs of trailing ones in the reverse order, if        any (0 for positive and 1 for negative): 0,1,1.    -   c. Encoding the remaining non-zero coefficients in reverse        order: 1,3. The non-zero coefficients are encoded as “1” with        Level VLC 0 table, and “0010” with Level VLC 1 table,        respectively.

Level VLC 0 Table Level VLC 1 Table

Level Code Level Code 1 1 1 10 −1 01 −1 11 2 001 2 010 −2 0001 −2 011 300001 3 0010 −3 000001 −3 0011 . . . . . . . . . . . .

-   -   d. Encoding total zeros (TotalZeros) from the beginning to the        last non-zero coefficients. The maximum total zeros is        16-NumCoeff. If NumCoeff is 16, TotalZeros must be 0. If        NumCoeff is 0, no further code is needed, i.e. the end of the        coding for this block. For the remaining 15 cases, each case        uses a Huffman table for coding TotalZeros (see TotalZero        table). In this case, TotalZeros is 3, which is coded as “1110”        with TotZeros table [NumCoeff=5].

TotZeros table NumCoef TotZeros 1 2 3 4 5 6 . . . 15 0 1 111 0010 11110101000 101100 . . . . . . 1 011 101 1101 1110 01010 101101 . . . . . . 2010 011 000 0110 01011 1010 . . . . . . 3 0011 001 010 1010 1110 001 . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . 15  000000100 —— — — — . . . . . .

-   -   e. Coding the positions of all zeros from the beginning to the        last non-zero coefficients. In this case, the positions of all        zeros are coded to “001101”.

The final encoded bits for this 4×4 block contain 25 bits: 0001011 | 011| 10010 | 1110 | 001101. One example of alternative values is theencoded bits of an entire encoded 4×4 block. Another example ofalternative value can be the bits coding signs of trailing ones asdescribed in the above step c). In such a case, changing the alternativevalue only changes the sign of trailing ones and therefore does notchange the total bit length of the encoded block.

Yet another example of encoded content is a JPEG2000 picture. Instead ofDCT transforms, JPEG2000 uses Digital Wavelet Transform (DWT). The DWTof a pixeled image is computed by the successive application of verticaland horizontal, low-pass and high-pass filters to the image pixels,where the resulting values are called ‘wavelet coefficients’. A waveletis an oscillating waveform that persists for only one or a few cycles.At each iteration, the low-pass only filtered wavelet coefficients ofthe previous iteration are decimated, then go through a low-passvertical filter and a high-pass vertical filter, and the results of thisprocess are passed through a low-pass horizontal and a high-passhorizontal filter. The resulting set of coefficients is grouped in four‘subbands’, namely the LL, LH, HL and HH subbands. Each iterationcorresponds to a certain ‘layer’ or ‘level’ of coefficients. The firstlayer of coefficients corresponds to the highest resolution level of theimage, while the last layer corresponds to the lowest resolution level.One example of alternative value for JPEG2000 encoded picture is theencoded LL subband at a specific layer (e.g. the lowest resolution).

A simple way to produce WUs is to use “trial and error” by performingthe following steps for each WU:

-   -   a) For each entropy code (e.g. a VLC in MPEG2), randomly select        a value as a candidate for an alternative value.    -   b) Determine if this candidate is a valid value by verifying the        format compliance after substituting this candidate for the        existing value. If not, go back to Step a).    -   c) Check the perceptual changes after exchanging the existing        value with this candidate. If the perceptual change is        acceptable, record this candidate as an alternative value for        the WU. Otherwise, go back to Step a).        The above steps are repeated until all needed WUs are produced.

In one embodiment, WUs can be produced with an exiting watermarkingsystem that works in transform domain. First with the selectedwatermarking system, produce two watermarked copies (A and B, both inencoded form) of an original encoded content with two different payloadinformation: the first copy contains the watermark payload in which eachbit is ‘1’ and the other copy contains the watermark payload in whicheach bit is ‘0. The steps 1 and 2 (see FIG. 1) can be performed in thefollowing manner to select and produce WUs by scanning A and B until thefollowing conditions are satisfied:

-   -   (Location selection) P1 a and P2 a are the start position and        end position of a plurality of consecutive encoded coefficients        (their positions before entropy coding are noted as C) in the        encoded content A, P1 b and P2 b are the start position and end        position of the same encoded coefficients in the encoded content        B, and (P2 b-P2 a) and (P1 b-P1 a) must be equal (and are equal        to length L), and    -   (Calculation of alternative values) Assume that at least one bit        of watermark payload information is to be embedded in the        encoded coefficients (in both A and B). Recording one watermark        unit (P,L,C,V,V1,V2) in the following manner: P=P1 a, L=P2 a-P1        a, V is not available in this case, V1 is the value from the        position P1 a to P2 a in A, and V2 is the value from the        position P1 b to P2 b in B.    -   To select N watermark units, repeat the second step above.

In an alternative embodiment, a bit of watermark payload is encoded as arelationship between two selected quantized coefficients in a middlefrequency. By scanning the encoded content a watermark unit (P, L, C, V,V1, V2, . . . Vn) can be located and determined with the followingconditions:

-   -   V contains a plurality of consecutive encoded coefficients and        there is at least one relationship (the kind/category of the        relationships are predefined) between and/or among these        coefficients. For example, if C1, C2 are two consecutive encoded        coefficients and a relationship exists C1>C2+T where T is a        threshold which may be used to adjust the strength of the        relationship, which in turn determine the watermark robustness.    -   According to the perceptual model of the watermarking system,        the coefficients can be modified to establish another        relationship without perceptual change of the content. For        example, C1 and C2 can be modified to C1′ and C2′ respectively        so that a reverse relationship is formed, i.e. C1′<C2′.        Recording the new value containing C1′ and C2′ as an alternative        value Vi (1<=i<=n). Repeat this step until all alternative        values are found.    -   To select N watermark units, repeat the above steps. Ideally the        first N watermark units containing the most coefficients are        selected so that most relationships can be potentially        established within each watermark unit. In the referenced        watermarking system, one relationship may represent one bit of        watermark payload therefore the watermark unit with most        relationships may have the highest capability of embedding more        bits of watermark payload.

In yet another example of a watermarking system, a bit of watermarkpayload is encoded as a relationship between property values of two setsof pixel values, or DCT/wavelet coefficients. Typical examples ofproperties include the average luminance, average color histogramdistribution, or energy in certain frequency subband.

One principle of the present invention is to pre-process encoded contentand generate alternative values containing watermarking signals. Thesealternative values will be later used to replace clearly defined partsof the encoded content. Although the existing watermarking systems canbe applied to pre-processing (i.e. Step 1 and 2) and actual insertion(Step 3), new watermarking algorithms can be developed specially forthis scheme.

A simple and generic watermark detection method is to correlate thecurrent coefficient in the watermarked content with the alternativevalues of a WU to find the alternative value having the best match tothe current coefficient. To perform such correlation, the alternativevalues need to be decoded to coefficient values by using thecorresponding entropy decoder. Each alternative value of a WU maycontain a watermark signal, which corresponds to watermark payloadinformation. By finding the matched alternative values in all WUs, thewatermark payload information can be retrieved. Watermark detection canbe performed at multiple levels. At the first level, P, L and C elementsof WUs are required. Assume the number of Wus is N. For each WU, obtainthe coefficient values according to the global coefficient indexes C. Ifthe input content is baseband or encoded in other format than the formatused in watermark embedding, transform the content first to the sameencoded form as the watermark embedding. The raw watermark payload bitsare retrieved by observing the relationships between and/or among thecoefficients, or between property values of pixel values or coefficientvalues. FIG. 3 is a flowchart of the watermark detection process of thepresent invention. At 305 an index is initialized. ‘N’ watermark unitsare accepted/received at 310 and the suspect content is received at 315.The index is tested at 320 to determine if all watermarks have beenprocessed/detected. If the index is greater than “N”, the number ofwatermarks then the process is complete. If the index is not greaterthen “N” then the process proceeds to 325 where the coefficients areobtained. At 330 the bit values are retrieved according to therelationships between/among the coefficients. At 335 the index isincremented and the process proceeds to step 320.

If very weak relationships are retrieved from a WU, it would be helpfulto access the other elements V and V1, V2, . . . Vn of the WU in orderto provide additional information to determine the relationship. Arelationship can be better determined by correlating the currentcoefficients with the coefficients encoded in V, V1, V2, . . . and Vn.The highest correlation value is the correlation indicating the closestmatch between the current coefficient and one of V, V1, V2, . . . Vn.That is, the highest correlation between the current coefficients andthe coefficients encoded in Vi (1≦i ≦n) indicates a match. For suspectcontent with serious attacks, the original content may be required forfurther analysis and correlation.

Another way to use WUs for delivering copyrighted content is to use WUsto recover the corrupted content prior to playback. In such a case, theencoded content that is delivered to a playback/rendering device isexplicitly corrupted either in the format/structure (invalid format) orin the content (degraded content). For a format-corrupted content, adecoder will not be able to decode it. If the corruptions are applied tothe content, this content can be decoded and played but the corruptedportions of the content can have various visual artefacts such as noise,warning, and random patterns.

To produce a format-corrupted content, with a given watermark unit (P,L, V, V1, V2, . . . Vn), one can simply replace the L bits starting at Pin the encoded content to be delivered with a random value. To furtherensure that the random value indeed corrupts the format, one can simplyexclude the random values, which are valid entropy codes. To introducevisual artefacts in the content or degraded content, select a validalternative value, which is far from the original value V and all otheralternative values V1, V2, . . . Vn, to the replace the current value inthe encoded content. One exemplary use of corrupted format or degradedquality is to countermeasure a specific device or one category ofdevices that have been compromised or used for content piracy. Thesealternative values that cause corrupted format or degraded content areselected for substitution only for a specific watermark payload, whichidentifies the device or the category of devices.

A watermark can be removed if WUs are known, by simply restoring thecontent to the original value V in each WU. To overwrite an existingwatermark in the encoded content, one can simply select an appropriatevalue from V1, V2, . . . Vn in each WU according to the payloadinformation to replace the value containing the existing watermark inthe encoded content. A removable watermark is useful in someapplications for supporting multi-generation watermarks. An example ofmulti-generation watermarks is to embed a forensic mark in each step ofcontent post-production. To avoid accumulation of perceptual degradationpotentially introduced by multiple watermarks, one may wish remove someor all previous watermarks before a new watermark is embedded. FIG. 4shows an example of the watermark inserter in a playback device. Thewatermark inserter embeds a watermark by replacing somevalues/coefficients in the encoded content received by the devices withthe alternative values specified in WUs. A replaceable watermark canalso be useful for switching the states of copy protection forcopyrighted content. For example, for a unit of content with a watermarkinformation indicating “copy once”, after one copy is made one may wishto change the watermark information from “copy once” to “no copy”. Thereare several approaches to avoid any potential conflict of multiplewatermarks (i.e. interfere between multiple watermarks). One approach isto use a unique subband for each generation of watermarks. Anotherapproach is to select WUs with the positions that are different from theprevious watermarks.

It is to be understood that the present invention may be implemented invarious forms of hardware, software, firmware, special purposeprocessors, or a combination thereof. Preferably, the present inventionis implemented as a combination of hardware and software. Moreover, thesoftware is preferably implemented as an application program tangiblyembodied on a program storage device. The application program may beuploaded to, and executed by, a machine comprising any suitablearchitecture. Preferably, the machine is implemented on a computerplatform having hardware such as one or more central processing units(CPU), a random access memory (RAM), and input/output (I/O)interface(s). The computer platform also includes an operating systemand microinstruction code. The various processes and functions describedherein may either be part of the microinstruction code or part of theapplication program (or a combination thereof), which is executed viathe operating system. In addition, various other peripheral devices maybe connected to the computer platform such as an additional data storagedevice and a printing device.

It is to be further understood that, because some of the constituentsystem components and method steps depicted in the accompanying figuresare preferably implemented in software, the actual connections betweenthe system components (or the process steps) may differ depending uponthe manner in which the present invention is programmed. Given theteachings herein, one of ordinary skill in the related art will be ableto contemplate these and similar implementations or configurations ofthe present invention.

1. A method for inserting a watermark in encoded content, said methodcomprising: receiving encoded content; receiving at least onepre-processed watermark unit; and replacing directly a number of bitsstarting at a position with an alternative value, wherein saidalternative value has embedded therein watermark signals, wherein saidalternative value is selected from among a plurality of alternativevalues specified in said at least one said watermark unit.
 2. The methodaccording to claim 1, wherein said number of bits is specified in saidat least one watermark unit.
 3. The method according to claim 1, whereinsaid position is specified in said at least one watermark unit. 4.(canceled)
 5. A system for inserting a watermark in encoded content,comprising: means for receiving encoded content; means for receiving atleast one pre-processed watermark unit; and means for replacing directlya number of bits starting at a position with an alternative value,wherein said alternative value has embedded therein watermark signals,wherein said alternative value is selected from among a plurality ofalternative values specified in said at least one said watermark unit.6. The system according to claim 5, wherein said number of bits isspecified in said at least one watermark unit.
 7. The system accordingto claim 5, wherein said position is specified in said at least onewatermark unit.
 8. (canceled)
 9. The system according to claim 5,wherein said system is a playback device, said playback device receivingencoded content and said at least one watermark unit, wherein saidencoded content and said at least one watermark unit are one of storedas part of said encoded content, stored as metadata and stored in adirectory.
 10. A method for inserting a watermark in encoded content,said method comprising: receiving encoded content; receiving at leastone pre-processed watermark unit; receiving a sequence of bits, whereinsaid sequence of bits comprises watermark payload information; andperforming one of replacing directly a number of bits starting at aposition with an alternative value and leaving unchanged said number ofbits starting at said position based on bit values of said sequence ofbits of said watermark payload information and wherein said alternativevalue has embedded therein watermark signals, wherein said alternativevalue is selected from among a plurality of alternative values specifiedin said at least one said watermark unit.
 11. A system for inserting awatermark in encoded content, comprising: means for receiving encodedcontent; means for receiving at least one pre-processed watermark unit;means for receiving a sequence of bits, wherein said sequence of bitscomprises watermark payload information; and means for performing one ofreplacing directly a number of bits starting at a position with analternative value and leaving unchanged said number of bits starting atsaid position based on bit values of said sequence of bits of saidwatermark payload information and wherein said alternative value hasembedded therein watermark signals, wherein said alternative value isselected from among a plurality of alternative values specified in saidat least one said watermark unit.
 12. A method for replacing a watermarkin encoded content, said method comprising: receiving encoded content;receiving at least one pre-processed watermark unit; and replacing anumber of bits starting at a position in said encoded content with avalue, wherein said value is specified in said at least one watermarkunit and further wherein said position is specified in said at least onewatermark unit, wherein said value is an alternative value selected fromamong a plurality of alternative values specified in said watermarkunit, wherein said selected alternative value forms a differentwatermark from that watermark which is being replaced.
 13. (canceled)14. A system for replacing a watermark in encoded content, said methodcomprising: means for receiving encoded content; means for receiving atleast one pre-processed watermark unit; and means for replacing a numberof bits starting at a position in said encoded content with a value,wherein said value is specified in said at least one watermark unit,wherein said number of bits is specified in said at least one watermarkunit and further wherein said position is specified in said at least onewatermark unit, wherein said value is an alternative value selected fromamong a plurality of alternative values specified in said watermarkunit, wherein said selected alternative value forms a differentwatermark from that watermark which is being replaced.
 15. (canceled)16. A method for watermark detection, said method comprising: receivingat least one pre-processed watermark unit; receiving watermarkedcontent; retrieving a plurality of coefficient values from thewatermarked content; and retrieving bit values of said watermark fromsaid coefficient values, wherein said bit values are retrieved bycorrelating a current coefficient to a plurality of coefficient valuescoded in said at least one watermark unit to determine a best match. 17.(canceled)
 18. The method according to claim 16, wherein said bit valuesare retrieved from said plurality of coefficient values according torelationships between/among said coefficient values.
 19. The methodaccording to claim 18, wherein said relationships are weak and a currentcoefficient is correlated to determine a match from among said pluralityof coefficient values coded in said at least one watermark unit.
 20. Asystem for watermark detection, comprising: means for receiving at leastone pre-processed watermark unit; means for receiving watermarkedcontent; means for retrieving a plurality of coefficient values from thewatermarked encoded; and means for retrieving bit values of saidwatermark from said coefficient values, wherein said bit values areretrieved by correlating a current coefficient to a plurality ofcoefficient values coded in said at least one watermark unit todetermine a best match.
 21. (canceled)
 22. The system according to claim20, wherein said bit values are retrieved from said plurality ofcoefficient values according to relationships between/among saidcoefficient values.
 23. The system according to claim 22, wherein saidrelationships are weak and a current coefficient is correlated todetermine a match from among said plurality of coefficient values codedin said at least one watermark unit.
 24. A method for generatingwatermark units, said method comprising: selecting a position where avalue in encoded content is to be replaced by an alternative value; andcalculating a plurality of alternative values for each selectedposition, wherein each of said alternative values contains a watermarksignal, further wherein each of said alternative values has a samenumber of bits as said value that a selected one of said plurality ofalternative value replaces, further wherein said replacement does notresult in perceptual changes to said encoded content, further whereinsaid replacement does not result in non-compliant format of said encodedcontent and wherein said method is performed by a pre-processor. 25.(canceled)
 26. (canceled)
 27. (canceled)
 28. (canceled)
 29. (canceled)30. An apparatus for generating watermark units, comprising: means forselecting a position where a value in encoded content is to be replacedby an alternative value; and means for calculating a plurality ofalternative values for each selected position, wherein each of saidalternative value contains a watermark signal, further wherein each ofsaid alternative values has a same number of bits as said value that aselected one of said plurality of alternative values replaces, furtherwherein said replacement does not result in perceptual changes to saidencoded content, further wherein said replacement does not result innon-compliant format of said encoded content and wherein said apparatusis a pre-processor.
 31. (canceled)
 32. (cancelled)
 33. (canceled) 34.(canceled)
 35. (canceled)
 36. The method according to claim 24, whereinsaid selected alternative value is an alternative value causing leastperceptual change.
 37. The apparatus according to claim 30, wherein saidselected alternative value is an alternative value causing leastperceptual change.
 38. An apparatus, comprising: means for storingencoded content; and means for storing watermark units.
 39. Theapparatus according to claim 39, wherein said watermark units areencrypted.
 40. The apparatus according to claim 39, wherein saidwatermark units are meta data of said encoded content.
 41. The apparatusaccording to claim 39, wherein said watermark units are steganographicdata hidden in one of syntax elements and said encoded content.
 42. Theapparatus according to claim 41, wherein access to said watermark unitsis controlled by a key.
 43. The apparatus according to claim 39, whereinsaid watermark units are stored as a separate file.
 44. The apparatusaccording to claim 39, wherein said apparatus is one of an optical disc,a tape, and a hard drive.
 45. An apparatus, comprising: means fortransmitting encoded content; and means for transmitting watermarkunits.
 46. The apparatus according to claim 44, wherein said watermarkunits are encrypted.
 47. The apparatus according to claim 44, whereinsaid watermark units are meta data of said encoded content.
 48. Theapparatus according to claim 44, wherein said watermark units aresteganographic data hidden in one of syntax elements and said encodedcontent.
 49. The apparatus according to claim 48, wherein access to saidwatermark units is controlled by a key.
 50. The apparatus according toclaim 44, wherein said watermark units are transmitted as a separatefile multiplexed with said encoded content.
 51. The apparatus accordingto claim 44, wherein said watermark units are encrypted and transmittedover a separate channel.